Distributional coding of associative learning in discrete populations of midbrain dopamine neurons.

Midbrain dopamine neurons are thought to play key roles in learning by conveying the difference between expected and actual outcomes. Recent evidence suggests diversity in dopamine signaling, yet it remains poorly understood how heterogeneous signals might be organized to facilitate the role of downstream circuits mediating distinct aspects of behavior. Here, we investigated the organizational logic of dopaminergic signaling by recording and labeling individual midbrain dopamine neurons during associative behavior. Our findings show that reward information and behavioral parameters are not only heterogeneously encoded but also differentially distributed across populations of dopamine neurons. Retrograde tracing and fiber photometry suggest that populations of dopamine neurons projecting to different striatal regions convey distinct signals. These data, supported by computational modeling, indicate that such distributional coding can maximize dynamic range and tailor dopamine signals to facilitate specialized roles of different striatal regions.

Cerebro-cerebellar networks facilitate learning through feedback decoupling.

Behavioural feedback is critical for learning in the cerebral cortex. However, such feedback is often not readily available. How the cerebral cortex learns efficiently despite the sparse nature of feedback remains unclear. Inspired by recent deep learning algorithms, we introduce a systems-level computational model of cerebro-cerebellar interactions. In this model a cerebral recurrent network receives feedback predictions from a cerebellar network, thereby decoupling learning in cerebral networks from future feedback. When trained in a simple sensorimotor task the model shows faster learning and reduced dysmetria-like behaviours, in line with the widely observed functional impact of the cerebellum. Next, we demonstrate that these results generalise to more complex motor and cognitive tasks. Finally, the model makes several experimentally testable predictions regarding cerebro-cerebellar task-specific representations over learning, task-specific benefits of cerebellar predictions and the differential impact of cerebellar and inferior olive lesions. Overall, our work offers a theoretical framework of cerebro-cerebellar networks as feedback decoupling machines.

Developmental depression-to-facilitation shift controls excitation-inhibition balance.

Changes in the short-term dynamics of excitatory synapses over development have been observed throughout cortex, but their purpose and consequences remain unclear. Here, we propose that developmental changes in synaptic dynamics buffer the effect of slow inhibitory long-term plasticity, allowing for continuously stable neural activity. Using computational modeling we demonstrate that early in development excitatory short-term depression quickly stabilises neural activity, even in the face of strong, unbalanced excitation. We introduce a model of the commonly observed developmental shift from depression to facilitation and show that neural activity remains stable throughout development, while inhibitory synaptic plasticity slowly balances excitation, consistent with experimental observations. Our model predicts changes in the input responses from phasic to phasic-and-tonic and more precise spike timings. We also observe a gradual emergence of short-lasting memory traces governed by short-term plasticity development. We conclude that the developmental depression-to-facilitation shift may control excitation-inhibition balance throughout development with important functional consequences.

Pre- and postsynaptically expressed spike-timing-dependent plasticity contribute differentially to neuronal learning.

A plethora of experimental studies have shown that long-term synaptic plasticity can be expressed pre- or postsynaptically depending on a range of factors such as developmental stage, synapse type, and activity patterns. The functional consequences of this diversity are not clear, although it is understood that whereas postsynaptic expression of plasticity predominantly affects synaptic response amplitude, presynaptic expression alters both synaptic response amplitude and short-term dynamics. In most models of neuronal learning, long-term synaptic plasticity is implemented as changes in connective weights. The consideration of long-term plasticity as a fixed change in amplitude corresponds more closely to post- than to presynaptic expression, which means theoretical outcomes based on this choice of implementation may have a postsynaptic bias. To explore the functional implications of the diversity of expression of long-term synaptic plasticity, we adapted a model of long-term plasticity, more specifically spike-timing-dependent plasticity (STDP), such that it was expressed either independently pre- or postsynaptically, or in a mixture of both ways. We compared pair-based standard STDP models and a biologically tuned triplet STDP model, and investigated the outcomes in a minimal setting, using two different learning schemes: in the first, inputs were triggered at different latencies, and in the second a subset of inputs were temporally correlated. We found that presynaptic changes adjusted the speed of learning, while postsynaptic expression was more efficient at regulating spike timing and frequency. When combining both expression loci, postsynaptic changes amplified the response range, while presynaptic plasticity allowed control over postsynaptic firing rates, potentially providing a form of activity homeostasis. Our findings highlight how the seemingly innocuous choice of implementing synaptic plasticity by single weight modification may unwittingly introduce a postsynaptic bias in modelling outcomes. We conclude that pre- and postsynaptically expressed plasticity are not interchangeable, but enable complimentary functions.

Learning offline: memory replay in biological and artificial reinforcement learning.

Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory.

A deep learning framework for neuroscience.

Systems neuroscience seeks explanations for how the brain implements a wide variety of perceptual, cognitive and motor tasks. Conversely, artificial intelligence attempts to design computational systems based on the tasks they will have to solve. In artificial neural networks, the three components specified by design are the objective functions, the learning rules and the architectures. With the growing success of deep learning, which utilizes brain-inspired architectures, these three designed components have increasingly become central to how we model, engineer and optimize complex artificial learning systems. Here we argue that a greater focus on these components would also benefit systems neuroscience. We give examples of how this optimization-based framework can drive theoretical and experimental progress in neuroscience. We contend that this principled perspective on systems neuroscience will help to generate more rapid progress.

Model-Based Inference of Synaptic Transmission.

Synaptic computation is believed to underlie many forms of animal behavior. A correct identification of synaptic transmission properties is thus crucial for a better understanding of how the brain processes information, stores memories and learns. Recently, a number of new statistical methods for inferring synaptic transmission parameters have been introduced. Here we review and contrast these developments, with a focus on methods aimed at inferring both synaptic release statistics and synaptic dynamics. Furthermore, based on recent proposals we discuss how such methods can be applied to data across different levels of investigation: from intracellular paired experiments to in vivo network-wide recordings. Overall, these developments open the window to reliably estimating synaptic parameters in behaving animals.

Computational roles of plastic probabilistic synapses.

The probabilistic nature of synaptic transmission has remained enigmatic. However, recent developments have started to shed light on why the brain may rely on probabilistic synapses. Here, we start out by reviewing experimental evidence on the specificity and plasticity of synaptic response statistics. Next, we overview different computational perspectives on the function of plastic probabilistic synapses for constrained, statistical and deep learning. We highlight that all of these views require some form of optimisation of probabilistic synapses, which has recently gained support from theoretical analysis of long-term synaptic plasticity experiments. Finally, we contrast these different computational views and propose avenues for future research. Overall, we argue that the time is ripe for a better understanding of the computational functions of probabilistic synapses.

Differential Regulation of Evoked and Spontaneous Release by Presynaptic NMDA Receptors.

Presynaptic NMDA receptors (preNMDARs) control synaptic release, but it is not well understood how. Rab3-interacting molecules (RIMs) provide scaffolding at presynaptic active zones and are involved in vesicle priming. Moreover, c-Jun N-terminal kinase (JNK) has been implicated in regulation of spontaneous release. We demonstrate that, at connected layer 5 pyramidal cell pairs of developing mouse visual cortex, Mg2+-sensitive preNMDAR signaling upregulates replenishment of the readily releasable vesicle pool during high-frequency firing. In conditional RIM1αß deletion mice, preNMDAR upregulation of vesicle replenishment was abolished, yet preNMDAR control of spontaneous release was unaffected. Conversely, JNK2 blockade prevented Mg2+-insensitive preNMDAR signaling from regulating spontaneous release, but preNMDAR control of evoked release remained intact. We thus discovered that preNMDARs signal differentially to control evoked and spontaneous release by independent and non-overlapping mechanisms. Our findings suggest that preNMDARs may sometimes signal metabotropically and support the emerging principle that evoked and spontaneous release are distinct processes. VIDEO ABSTRACT.

Synaptic Transmission Optimization Predicts Expression Loci of Long-Term Plasticity.

Long-term modifications of neuronal connections are critical for reliable memory storage in the brain. However, their locus of expression-pre- or postsynaptic-is highly variable. Here we introduce a theoretical framework in which long-term plasticity performs an optimization of the postsynaptic response statistics toward a given mean with minimal variance. Consequently, the state of the synapse at the time of plasticity induction determines the ratio of pre- and postsynaptic modifications. Our theory explains the experimentally observed expression loci of the hippocampal and neocortical synaptic potentiation studies we examined. Moreover, the theory predicts presynaptic expression of long-term depression, consistent with experimental observations. At inhibitory synapses, the theory suggests a statistically efficient excitatory-inhibitory balance in which changes in inhibitory postsynaptic response statistics specifically target the mean excitation. Our results provide a unifying theory for understanding the expression mechanisms and functions of long-term synaptic transmission plasticity.

Functional consequences of pre- and postsynaptic expression of synaptic plasticity.

Growing experimental evidence shows that both homeostatic and Hebbian synaptic plasticity can be expressed presynaptically as well as postsynaptically. In this review, we start by discussing this evidence and methods used to determine expression loci. Next, we discuss the functional consequences of this diversity in pre- and postsynaptic expression of both homeostatic and Hebbian synaptic plasticity. In particular, we explore the functional consequences of a biologically tuned model of pre- and postsynaptically expressed spike-timing-dependent plasticity complemented with postsynaptic homeostatic control. The pre- and postsynaptic expression in this model predicts (i) more reliable receptive fields and sensory perception, (ii) rapid recovery of forgotten information (memory savings), and (iii) reduced response latencies, compared with a model with postsynaptic expression only. Finally, we discuss open questions that will require a considerable research effort to better elucidate how the specific locus of expression of homeostatic and Hebbian plasticity alters synaptic and network computations.This article is part of the themed issue 'Integrating Hebbian and homeostatic plasticity'.

Unified pre- and postsynaptic long-term plasticity enables reliable and flexible learning.

Although it is well known that long-term synaptic plasticity can be expressed both pre- and postsynaptically, the functional consequences of this arrangement have remained elusive. We show that spike-timing-dependent plasticity with both pre- and postsynaptic expression develops receptive fields with reduced variability and improved discriminability compared to postsynaptic plasticity alone. These long-term modifications in receptive field statistics match recent sensory perception experiments. Moreover, learning with this form of plasticity leaves a hidden postsynaptic memory trace that enables fast relearning of previously stored information, providing a cellular substrate for memory savings. Our results reveal essential roles for presynaptic plasticity that are missed when only postsynaptic expression of long-term plasticity is considered, and suggest an experience-dependent distribution of pre- and postsynaptic strength changes.

Target-cell-specific short-term plasticity in local circuits.

Short-term plasticity (STP) denotes changes in synaptic strength that last up to tens of seconds. It is generally thought that STP impacts information transfer across synaptic connections and may thereby provide neurons with, for example, the ability to detect input coherence, to maintain stability and to promote synchronization. STP is due to a combination of mechanisms, including vesicle depletion and calcium accumulation in synaptic terminals. Different forms of STP exist, depending on many factors, including synapse type. Recent evidence shows that synapse dependence holds true even for connections that originate from a single presynaptic cell, which implies that postsynaptic target cell type can determine synaptic short-term dynamics. This arrangement is surprising, since STP itself is chiefly due to presynaptic mechanisms. Target-specific synaptic dynamics in addition imply that STP is not a bug resulting from synapses fatiguing when driven too hard, but rather a feature that is selectively implemented in the brain for specific functional purposes. As an example, target-specific STP results in sequential somatic and dendritic inhibition in neocortical and hippocampal excitatory cells during high-frequency firing. Recent studies also show that the Elfn1 gene specifically controls STP at some synapse types. In addition, presynaptic NMDA receptors have been implicated in synapse-specific control of synaptic dynamics during high-frequency activity. We argue that synapse-specific STP deserves considerable further study, both experimentally and theoretically, since its function is not well known. We propose that synapse-specific STP has to be understood in the context of the local circuit, which requires combining different scientific disciplines ranging from molecular biology through electrophysiology to computer modeling.